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§  1.  Introduction 

The  problem  considered  in  this  paper  is  whether  the  capacity  of 
a  discrete  time  Gaussian  channel  (GO  is  increased  by  feedback  or  not. 

It  is  known  that  the  capacity  of  a  white  GC  is  not  increased  by 
feedback  (see  [6]  for  continuous  time  case  and  Theorem  2  of  this  paper 
for  discrete  time  case).  On  the  other  hand,  some  examples  of 
non-white  GC’s  have  been  presented  to  show  that  the  capacity  can  be 
increased  by  feedback  (see  C4]  for  continuous  time  case  and  [3],  [5] 
for  discrete  time  case). 

The  main  aim  of  this  paper  is  to  give  conditions  under  which  the 
capacity  of  a  discrete  time  GC  is  increased  by  feedback.  The  GC  to 
be  considered  is  presented  by 

+  Z„,  n  €  N  5  (!,•••, N>,  (1) 

n  n  n 

where  X  ,  Y  and  Z_  represent  the  channel  input,  the  output  and 
n  n  n 

the  noise,  respectively,  at  time  n,  and  N  is  the  terminal  time. 

The  noise  Z  =  (Z  ;  n  €  M)  is  a  Gaussian  process  such  that  Z  ,  n  € 

n  n 

N,  are  linearly  independent.  We  assume  that  an  average  power 
constraint 

N  „  , 

2  E[X„3  i  p^,  (2) 

n=l  " 

is  imposed  on  the  input  X  =  (X_} ,  where  p  >  0  is  a  given  constant. 

n 

Let  9  be  a  message,  to  be  transmitted,  which  is  a  random  variable 

or  a  stochastic  process  independent  of  the  noise  Z.  Denote  by  7(6) 

and  7  (Y)  the  a-fields  generated  by  0  and  (Y  ;  n  <  m> , 
m  n 


-  U  - 


respectively.  The  GC  (1)  is  with  feedback  if  is  a  function  of 

the  message  and  the  channel  output  up  to  n  -  1,  namely 

(a.l)  is  7(6)  V  7  , (Y)  measurable, 

n  n-l 

The  GC  is  without  feedback  if  X^^  is  a  function  of  the  message, 
namely 

(a. 2)  X^  is  7(0)  measurable, 
n 

2  2 

In  each  case  we  denote"  by  )  and  ^q(P  )  the  classes  of  all 

admissible  pairs  of  a  message  and  an  input: 


((0,X); 

0  and  X 

satisfy  (2)  and 

(a.l)} ; 

((0,X) ; 

0  and  X 

satisfy  (2)  and 

(a.2)}  . 

We  denote  by 

I (0.Y) 

the  mutual  information 

between 

the 

message 

and  the  output 

Y  =  (Y„ 
n 

;  n  €  W} . 

Under  the  constraint 

(2) 

,  the 

2  2 

capacities  Cp(p  )  and  Cq(p  )  of  the  GC  (1)  with  feedback  and 
without  feedback,  respectively,  are  defined  by 

Cp(p^)  =  sup  (1(0, Y);  (0,X)  €  iJp(P^)> 

and 

Cq(p^)  =  sup  (1(0, Y);  (0,X)  € 

Note  that  if  the  GC  is  without  feedback  we  can  identify  the  message 
with  the  input  and  that 

Cq(p^)  =  sup  (I(X,Y);  X  €  Sq(p^)}, 

2 

where  ^q(P  )  ~  (X;  X  satisfies  (2)  and  is  independent  of  Z) . 

If  the  GC  is  without  feedback,  a  formula  for  the  capacity 


Cq(p^)  is  known  (see  Theorem  1).  If  Z  =  {Z^^)  is  a  white  noise, 
namely  if  Z^,  n  €  N,  are  mutually  independent,  the  GC  is  called  a 
white  Gaussian  channel  (WGC).  Since  it  is  known  that  the  capacity 
of  the  WGC  is  not  increased  by  feedback,  we  are  concerned  with  the 
case  of  the  non-white  GC. 

Let  2:  =  ^  be  the  covariance  matrix  of  Z.  We  say 

that  the  Gaussian  process  Z  =  iZ^i  n  €  N)  is  blockwise  white,  if 

n 

i\t\ 

there  exist  matrices  2  ,k=l,***,K  (1<K<N  and 

^k~l  ®k  ”  such  that 


Z  - 


(3) 


The  relation  ~  is  defined  as  follows.  For  NxN  matrices  A  = 

(»mn)m.n6W  ®  '  (  “mn  )n.,n6lN-  If  there 

exists  a  permutation  n  on  N  =  (1,***,N)  such  that  b _  » 

mn 

n  €  N.  We  note  that  if  Z  is  blockwise  white  then 

7i(m)7t(n) 

Z  =  {Z_,  N.  ,  <  n  i  N^> ,  k  =  !,*••, K,  are  mutually 
independent,  where  Nj^  =  m^  +  •••♦  mj^.  The  GC  is  said  to  be 
blockwise  white  if  the  noise  Z  is  blockwise  white.  If  Z  is  not 
blockwise  white,  Z  is  called  a  completely  non^white  noise  and  the 
corresponding  GC  is  also  called  a  completely  non-white  GC. 

It  will  be  shown  that  the  capacity  of  the  GC  is  increased  by 

feedback  if  the  GC  is  completely  non-white  or  the  GC  is  blockwise 

2  '  2 
white  but  not  white  and  the  power  p  is  greater  than  a  constant  p^ 

specified  later  (Theorem  4).  It  is  conjectured  that  if  the  GC  is 

2  2 

blockwise  white  and  p  <  p^  the  capacity  of  the  GC  with  feedback  is 
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the  same  as  the  capacity  of  the  same  channel  without  feedback.  We 
can  give  an  example  of  a  blockwise  white  GC  whose  capacity  is  not 
increased  by  feedback  (Theorem  5). 

It  is  known  that  the  capacity  of  the  GC  with  feedback  is  never 
more  than  twice  the  capacity  of  GC  without  feedback  [73.  We  have 
given  in  Part  I  [53  an  example  of  a  GC  whose  capacity  is  almost 
doubled  by  feedback. 


§  2.  Preliminaries 

This  section  is  devoted  to  summarizing  previously  known  results 
on  the  capacity  of  GC,  which  will  be  used  later. 

The  mutual  information  I(5,n)  between  random  variables  5  and 
n  is  defined  by 

if  the  joint  probability  distribution  of  K  and  n  is 

absolutely  continuous  with  respect  to  the  product  measure  of 

the  probability  distributions  and  of  ^  and  n, 

respectively,  where  d?^^/dM^XM^  is  the  Radon-Nikodym  derivative; 
otherwise  I(?,n)  is  infinite.  The  differential  entropy  h(5)  of 
an  m-dimens i onal  random  variable  K  with  a  continuous  distribution  is 
defined  by 
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h(5) 


log  p^(x)  dx, 


if  the  integral  exists,  where  p^(x)  is  the  probability  density 
function  of  If  h(5,n)  (=  h((^,n)>)  and  h(n)  exist,  the 
quantity  h(^ln)  defined  by 

h(^ln)  =  h(^,n)  -  h(n) 


is  called  the  conditional  entropy  of  5  given  n. 

The  following  properties  of  the  mutual  information  and  the 
differential  entropy  are  used  later. 


LEMMA  1.  Let  6,  X  =  {X„>  and  Y  =  lY^)  he  a  message,  an 

n  n 

input  and  the  corresponding  output  of  the  GC  (1),  respectively . 

N 

(i)  1(0, Y)  =  2  {h(Y„lY.,---,Y„  .)  -  h(Y„l0,Y. , • • • ,Y„  ,)> 

_  ,  n  1  n”i  n  i  n~i 

n*i 

N 

=  2  {h(Y„lY. .♦••,Y„  ,)  -  h(Z„lZ, , • • • ,Z„  ,)>,  (4) 

_  ,  n  1  n~  1  n  1  n- 1 

n= 1 

where  h(Y„ I Y. , • • • , Y„  ,)  =  h(Y,)  if  n  =  1. 
n  1  n~i  1 

2 

(ii)  Denote  by  the  variance  of  the  conditional  distribution 

of  Y  given  (Y, ,***,Y„  Then 

n  1  n“  1 

hCYpI Yi , • • • ,Yn_i)  ^  2  (2neTj).  (5) 

\ 

The  equality  holds  in  (5)  if  and  only  if  the  conditional  distribution 
is  Gaussian. 

(iii)  If  (X,Z)  =  {X„,Z„;  n  €  N>  is  Gaussian,  then 

n  n 
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1(0, Y)  =  ^  (log  iryl  -  log  III), 

where  is  the  covariance  matrix  of  Y  =  (Y  >  and  I • I  denotes 

1  n 

the  determinant. 

2 

The  capacity  Cq(p  )  of  the  GC  (1)  without  feedback  is  known  to 

2 

be  determined  by  the  e^igenvalues  (A^;  n  €  IN)  of  the  covariance 

n 

matrix  Z  of  the  noise  Z  =  (Z  ) . 

n 

THEOREM  1  (see,  e.g.  [1]).  The  capacity  Cq(p^)  of  the  GC  (1) 
without  feedback  subject  to  (2)  is  given  by 

C-(p^)  =  ^  2  log  max  (1,  a^x^^), 

where  asa(p)>0  is  a  constant  uniquely  determined  by 
M  0  2  2 

2  max  (0,  a^  -  xf)  =  p^.  (6) 

_ 1  n 


There  exists  an 

^PZP  =  A,  where  A 

transposed  matrix  of 
defined  by 


orthogonal  matrix  P  =  I  p _ ]  such 

V  mn  ym,ne8\i 


^  x2 


4 


and  V  (=  P"^)  is  the 


P.  A  Gaussian  process  S  =  (5  ;  n  €  N) 

n 


that 


N 

5  =  ZP  (5„  =  2  P„„Z„,  n  €  N)  (7) 

n  _  ,  mn  ro 


is  a  whi te  noise. 


Indeed,  the  covariance  matrix  of  %  is  the 


diagonal  matrix  A.  Define  constants  -t  0,  n  €  N,  by 

2  2  2 

=  max  (0,  a  -  A^).  (8) 

n  n 

Let  ^  n  €  N>  be  a  zero  mean  Gaussian  process  independent  of 

5,  such  that  ®nd  n  =  (n_;  n  €  N>  be  defined  by 

m  n  mn  n  n 

=  €„  +  5^.  n  €  N. 

n  n  0“ 

We  define  Gaussian  processes  X®=  (X®;  n  €  N)  and  Y®=  {Y®;  n  €  N> 

n  n 

by 

N 

X°=  (X°  =  2  (9) 

n  _  ,  nm  m 
m=i 

and 

„  €  N. 

Then  X^  is  independent  of  Z  and  ,  EC(X°)^]  =  ,  Et§^]  *  . 

n- 1  n  n*  x  n 

In  the  case  of  without  feedback,  it  is  known  that  X^  is  the  optimal 
input  signal  in  the  sense  that  the  capacity  is  attained  by 
transmitting  X®: 

N 

C.(p^)  =  y  I  log  max  (^.a^xl^)  =  I(?,n)  =  I(X°,Y®).  (10) 

^  ^  n=l  " 

We  put  =  y„„(P)  =  ECX^X®]  and  denote  by  T  =  f  y  1 

mn  mn  ro  n  '  V.  mn  >m,n€N 

the  covariance  matrix  of  X^  =  (X®)  -m.  Since  a  =  a(p)  and  x  = 

n  n€W  n 

X  (p)  are  monotone  non-decreasing  in  p,  y  =  2*'^  ,  is  also 

n  mn  m=i  mn  m 

non-decreasing  in  p.  We  assume,  for  a  moment,  that  the  noise  Z  is 
blockwise  white  but  not  white.  Then  there  exists  a  permutation  n 

on  N  such  that  Z"*'"  =  Nj^.^  <  n  i  Nj^} ,  k  =  1 ,  •  •  • , K,  (0  =  Nq 
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<  Nj  < 


<  N„  =  N  and  1  <  K  <  N)  are  mutually  independent.  Here 


we  may  assume  that 


mj^  =  1 ,  k  i  L , 


mj^  k  2  and  Z 


(11) 


is  completely  non-white,  k  >  L, 


where  ~  ^k  ~  ^k-1  0  s:  L  <  N.  In  this  case  we  can  easily 

show  that 


’n(n,)7t(n)  =  “  Vl  <  ^  N,,'  Vl  <  »  ^  (H  t) 


P7i(m)K(n)  Jn,  ,<miN,  orthogonal  matrix.  Hence  we  have 


n>=N._,>l  "  m.n=N- 


=  2 


'n(n)  • 


(12) 


We  define  a  constant  i  0  by 


P-  =  inf  (p;  a^(p)  i  min 
°  n>NL 


(13) 


From  (8)  and  (12)  we  see  that  p  >  p^  if  and  only  if 


"'VI 


’’>t(n)7t(n)  ^  “• 


Thus  we  have  obtained  the  following 


LEMMA  2.  Let  Z  =  (Z^l  be  a  non-uhite  and  blockiiise  white 

n 

noise  satisfying  (11).  Then,  if  p  >  p^,  there  exists  an  integer 
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k  such  that  “  ’'^k-l  ^  ^ 


\ 

2 

n=N 


k-1 


+  1 


(14) 


When  the  GC  (1)  is  white,  the  following  theorem  is  known. 


THEOREM  2.  If  the  GC  (1)  is  vhite,  then  the  capacity  under  the 
constraint  (2)  is  not  increased  by  feedback: 

Cp(p^)  =  Cq(p^)  =  2  2  log  max  (1, 

2  2 

where  =  ECZ^]  and  a  is  given  by  (6>. 


For  the  completeness  we  shall  give  the  proof  in  Appendix. 

We  do  not  have  explicit  formulae  for  the  capacity  with  feedback 
except  for  the  case  of  WGC.  However  it  has  been  known  that  the 
capacity  is  attained  in  a  Gaussian  scheme.  We  consider  the 
following  conditions. 

(b.O)  A  message  0  =  {0„;  n  €  N)  is  a  white  Gaussian  noise  with 

n 

E[0„]  =  0  and  E[0^]  =  1. 
n  n 

(b.l)  An  input  X  =  {X„;  n  €  W)  is  of  the  form 

n 


Where  a.,  k  =  l,***,n,  are  constants  and  EC0.I7^  ,(Y)]  denotes  the 
K  K  n~  1 

conditional  expectation. 
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THEOREM  3  [2].  It  holds  that 


Cp(p^)  =  sup  {1(0, Y);  (0,X)  €  S(p^)>. 
vhere  «(p^)  =  {(0,X)  €  jrfp(p^);  0  and  X  satisfy  (b.O)  and  (b.l)>. 


§  3.  Main  Results 

In  this  section  we  shall  give  the  statements  of  our  main  results. 
The  proof  of  them  will  be  given  in  §  4. 

First,  we  give  conditions  for  the  increase  of  the  capacity  by 
feedback. 

THEOREM  4.  If 

(i)  the  noise  Z  of  the  GC  (i)  is  completely  non-vhite, 
or 

(ii)  the  noise  Z  is  blockuise  vhite  but  not  white  and  p  >  p^, 
where  Pq  is  the  constant  given  by  (13), 

then  the  capacity  of  the  CC  subject  to  (2)  is  increased,  by  feedback: 

Cq(P^)  ^  Cp(p^).  (15) 

2 

By  (10)  the  capacity  Cq(P  )  of  the  GC  without  feedback  is 

achieved  by  transmitting  the  signal  X®=  {X?}  of  (9).  To  prove 

n 

Theorem  4,  using  the  signal  X^  and  feedback,  we  shall  construct  a 
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coding  scheme  which  enables  us  to  transmit  mutual  information 

2 

strictly  greater  than  C^Cp  ). 

If  the  dimension  N  of  the  GC  (1)  is  equal  to  2,  a  non-white 
Gaussian  noise  Z  =  is  completely  non-white.  Therefore,  as 

a  consequence  of  Theorem  2  and  Theorem  4,  we  have  the  following 

COROLLARY.  The  "capacity  of  a  2 -dimensional  GC  is ^increased  by 
feedback  if  and  only  if  the  GC  is  not  vhite. 


It  is  conjectured  that  if  p  <  the  capacity  of  a  blockwise 

white  GC  is  not  increased  by  feedback.  Although  we  have  not 
succeeded  in  proving  this  for  the  general  case,  we  can  show  that 
there  exists  an  N  3)  dimensional  blockwise  white  GC  whose 
capacity  is  not  increased  by  feedback.  We  consider  a  3-dimensional 
non-whi te  GC 


"  =  1.2,2,  (16) 

n  n  n 

to  get  an  example  of  a  GC  whose  capacity  is  not  increased  by  feedback. 
2 

We  denote  by  n  =  1,2,3,  the  eigenvalues  of  the  covariance 

matrix  2  =  |  i  o  o  of  the  noise  Z  =  (Z.  ,Z..  .Z.) .  We 

V  mn  /m,n=i,4,o  i  z  j 

assume  that  Z^  is  independent  of  <Z^,Z2>  so  that 


^13  ■  ^23 


0. 


2 

We  may  take 

2 

and  A2  as 

1  1  ,  o  • 

Then  A?  = 

^  mn  y  m,  n=l , 2 

3 

2^.2 

^2  .2 

A)  i  A2 

ip  ♦  ^3 

the  eigenvalues  of  the  matrix 
We  assume  that 


(17) 


(18) 


and 


% 


°11  ^  <^22  *  ^  ,  _ L_ 

^  2 

A  p  +  a 

2 

are  satisfied,  where  A  =  <^^^22  ~  ^12*  Note  that  if 

2 

are  large  enough,  and  is  small  enough,  then 

large,  and  (16)  and  (19)  hold. 


33 


and 


(19) 


and 

2 


22 


are 


THEOREM  5.  Assume  that  the  conditions  (17)  -  (19)  are 

satisfied.  Then  the  capacity  of  the  3-dimensional  non-uhite  GC  (16) 
subject  to  (2)  is  not  increased  by  feedback. 


In  order  to  prove  Theorem  5  we  need  to  use  a  result  from  the 
capacity  of  a  2-dimensional  GC 

Y,.  =  +  Z^,  n  =  1,2,  (20) 

n  n  n 

where  Z  s  (Z^,Z2)  is  a  2-dimensional  zero  mean  Gaussian  random 

variable  with  covariance  matrix  Z  =  I  )».  «  i  o  satisfying  A  = 

V  mn  y m, n=i , z 

2  2  2 

^11^22  ~  ^12  ^  denote  by  Cp(Pj,P2)  the  capacity  of  the  GC 

(20)  with  feedback  under  the  constraint 

ECxJ]  ^  pj.  n  =  1,2,  (Pj^  i  0).  (21) 

We  can  show  the  following  result. 

THEOREM  6.  Uvder  the  constraint  (21),  the  capacity  of  the 
Z-dimensional  GC  (20)  with  feedback  is  given  by 
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2  2 

Cp(p‘.p‘) 


.  X 


log 


1  + 


’lA  *  °220l  ♦  “14  * 


(22) 


Moreover  it  holds  that 


Cp(p^)  =  sup  Cp(p^,P2). 


(23) 


uhere  the  suprenun  is  taken  for  all  p^  >:  0  and  p^  ^  0  such  that 

2  2  2 

Pi  +  P2  =  P  . 


§  4.  Proof  of  Theorems 

We  recall  that  the  capacity  Cq(p^)  of  the  GC  (1)  without 

feedback  is  attained  by  transmitting  the  signal  X®  =  (X®)  given  by 

n 

(9),  and  the  covariance  of  X®  is  denoted  by  =  y  (p)  =  ECX^X®]. 

mn  mn  m  n 

In  order  to  prove  Theorem  4  we  prepare  a  lemma. 

LEMMA  3.  Assume  that  (i)  or  (ii)  of  Theorem  4  hold.  Then 

there  exist  m  and  n  (n  ^  m)  such  that  y  ;  ^  0. 

mn 

PROOF.  At  first  we  assume  (i).  Let  us  suppose  that  y  =  0 
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for  any  m  ^  n.  Let  L  be  the  subset  of  N  such  that  ^  ^ 

L  and  X®  =  0,  n  €  M  =  W  N  L.  Since  X°  =  and  5  «  X®P  we 

have 

J([X°;  n  €  N]  =  jKCX°;  n  €  L]  =  AC?  ;  ^  0], 

n  n  n  n 

where  i([X.:  n  €  IN]  denotes  the  set  of  all  linear  combinations  of 
n  - 

X  ,  n  €  N.  Therefore  we  may  assume  that 
n 

0,  n  €  L,  and  =  0,  n  €  M.  (24) 

n  n 

It  follows  from  (24)  that 


=0  if  n  €  M  and  m  €  L. 
nm 

Using  (7)  and  (25)  we  have 


(25) 


N 

=  2 
m^l 


P._5_ 
nm  m 


*  2 

m€M  ™ 


€  m  €  M],  n  €  M, 

m 


and  iftZ  ;  n  €  M]  c  itC5  ;  n  €  M].  Since  Z„,  n  €  N,  are  linearly 
n  n  n 

independent,  it  should  be  held  that 


M.IZ\  n  €  M]  =  n  €  M] . 

n  n 

Therefore  we  see  that 


n 


N 

2  Pm„Z-  =  2  P«-Z  . 

1=1  m€lM  “ 


n  €  M, 


and  that 

p _ =  0  if  m  €  L  and  n  €  M. 

mn 

Thus,  If  n  €  L,  Independent  of  ro  €  M,  and 
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consequently  independent  of  Z^t  m  €  M.  Since  the  noise  Z  is 

completely  non-white,  this  means  that  L  »  M.  Therefore,  C  ^0 

n 

and  =  a^  -  >  0  for  all  n  €  W.  Thus,  for  any  m 

n,  we  have 


^mn  “  ^mk^nk^k  “  ^mk^nk^*  ~  ^k^ 


2  N  N  2 

®  ^mk**nk  ”  ^mk**nk^k 


*^mk**nk^k 


But  this  does  not  occur  unless  Z  is  a  white  noise,  and  contradicts 

the  assumption  (i).  Therefore  we  conclude  that  there  exist  m  and 

n  (m  n)  such  that  ^  0.  Secondly  we  assume  (ii).  In  this 

mn 

(ki 

case,  by  Lemma  2,  we  know  that  there  exists  k  such  that  Z  = 
^^n(n)*  ^'^k-l  ^  ^  completely  non-white,  i  2  and 

(14)  holds.  Therefore,  as  we  have  shown  above,  there  exist  Nj^_j  < 
ro  <  n  i  Nj^  such  that  ■>'^(n,)7t(n)  ^  ®*  Q.E.D. 


We  are  now  in  a  position  to  prove  Theorem  4. 

PROOF  OF  THEOREM  4.  Let  X®  =  {X®>  be  the  optimal  input 

n 

signal  given  by  (9).  By  Lemma  3,  there  exist  k  and  £  (1  ^  k  <  4 
i  N)  such  that  =  ECX®X®]  ^  0.  For  simplicity,  we  put 


We  define  a  message  0  =  ^®n^n€N 
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e„  =  X^.  n  t, 
n  n 


0^  =  X"  +  Xq, 


where  X^  is  a  zero  mean  Gaussian  random  variable  with  variance  0 
which  is  independent  of  X°  and  Z.  Then  we  define  an  input  signal 

^  =  <^n>n€N 

X  =  0  ,  n  t,  X.  =  0-  -  oeY.  , 
n  n  {.  i.  K 

where  Y  =  (Y  )  is  the  corresponding  output: 
n 


f  Y„  =  X^  ♦  =  0„  ♦  Z„.  n  9^  t, 

n  n  n  n  n 

,  Y-  =  X-  +  Z,  =  <0,  -  aY.  )  +  Z.. 


From  the  definition.  ECX^]  =  EC0^]  *  n  and 

E[xJ]  =  E[(0^  -  a(0j^  + 
s  ECXq]  +  EC(xJ  - 


=  0  >  y^^  -  2ayj^^  ♦  a  (y^,^  ♦  «  y^^. 


Hence 


N  «  N  , 

2  E[X„]  =  2  y__  =  P  . 
n=l  "  n=l  "" 


Thus  (0,X)  €  irfp(P  )  and  consequently 


1(0, Y)  i  Cp(p‘). 


(26) 


We  define  Y  =  by 


=  0„  ♦  Z„.  n  €  N, 

n  n  n 
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Then  it  is  clear  that  MIY:  n  €  IJ]  *  ACY  ;  n  €  N]  and 

n  n 

1(0, Y)  =  1(0, Y).  (27) 

By  Lemma  1,  I(0,Y)  is  given  by 

1(0, Y)  =  ^  (log  IQI  -  log  IZI),  (28) 

where  Q  *  (  is  ih®  covariance  matrix  of  Y.  We  denote 

by  Q®  =  ^  the  covariance  matrix  of  the  output  Y®  =  (Y®) 

given  by  Y°  =  X®  +  Z^.  Then  =  E[Y^Y„]  =  EIyJy®]  =  q°  for 

n  n  n  mn  m  n  m  n  mn 

any  (m,n)  ^  it,t)  and  q^^  =  E[(Y^)^]  =  Et(Yj)^]  +  EIX^]  *  qj^  +  d. 
Therefore 

IQI  =  |Q°I  +  >  IQ°I,  (29) 

where  is  (Itt)  cofactor  of  Q®.  It  follows  from  (28),  (29) 

and  (10)  that 

1(0, Y)  >  ^  {log  IQ®I  -  log  IZl}  =  I(X®,Y®)  =  Cq(p^).  (30) 

Thus  we  obtain  the  desired  inequality  (15)  from  (26),  (27)  and  (30). 

Q.E.D. 

PROOF  OF  THEOREM  5.  By  (18)  and  Theorem  1  we  know  that  the 
2 

capacity  Cq(p  )  without  feedback  is  given  by 

“  2  ^  *  ^^^3^  )• 

2 

We  proceed  to  calculate  the  capacity  Cp(p  )  of  the  GC  (16)  with 
feedback.  Let  (0,X)  be  any  admissible  pair  of  a  message  and  an 
input  signal,  namely  0  and  X  s  (Xj,X2,X2)  satisfy  (2)  and  (a.l). 


-  20  - 


We  put  =  E[X^],  n  =  1,2,3.  We  may  assume  that  ^2  *  ^3  ~ 


p  .  Using  Lemma  1  we  have 


1(0, Y)  =  I(0,(Yj,Y2))  ♦  h(YglYj,Y2)  -  h (Yg I  0 , Yj , Y2 ) 


=  I(0,(Yj,Y2))  ♦  h(YglYj.Y2)  -  ^  log  (27ieXg). 


(32) 


Since  Zg  is  independent  of  (Y^.Yg).  Zg  is  independent  of  Xg  and 


it  follows  from  (ii)  of  Lemma  1  that 


h(YglYj,Y2)  i  ^  log  {  2ne  (Pg  ♦  Ag)  }. 


(33) 


Clearly  I(0,(Yj,Y2))  is  upper  bounded  by  Cp(Pj,P2).  We  put 
Pj  +  Pg  =  -  P3.  then  pj.  Pg.  2p,p2  ^  ^  P^ .  Using  (22)  we 

get  the  following  inequality: 

I(0,(Yj,Y2))  ^  Cp(pJ,p^) 


^  2  ^og 


♦  P  ♦  Icr.ol 


1  +  <^T}p^ 


y2  . 


It  follows  from  (34)  and  (19)  that 


i<e,(Yj.Yj))  ii  log  (1 


(34) 


2  2 
P  + 


^  2  log  *  32 

^3  ^^3 


3  ''3 


(35) 


Combine  (31)  -  (33)  and  (35)  to  get 
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1(0, Y)  i  ^  log  [ 


n2  *  ,2 

Po  +  A« 


/>2  *  ,2 
P  ♦  Ao 


■  .^.3  ] 

*  A?  J 


«2  *  ,2 

^3  ^3 


=  ^  log  (  1  +  p^Ag^  ]  = 

9  9 

for  any  admissible  pair  (0,X).  This  means  that  C„(p  )  ^  C-(p  ). 

r  0 

Since  Cq(p^)  i  Cp(p^)  by  definition,  Cp(p^)  is  equal  to  Cq(p^). 

Q.E.D. 


PROOF  OF  THEOREM  6.  We  consider  the  following  coding  scheme 


Xi  =  PjOj 

Xg  =  aOg  +  fl(0^  -  0j). 


(36) 


where  6^  and  Sg  are  mutually  independent  zero  mean  Gaussian 
random  variables  with  variance  one,  0^  =  E[0jiy(Yj)],  and  a  and  fi 
are  constants  such  that 


♦  0^  ECI0J  -  0j|2]  *  pI 


(37) 


In  [2]  it  is  shown  that 


Cp(pj,p2)  =  sup  1(0, Y), 
o( ,  0 

where  0  =  (0^,02),  Y  =  (Yj,Y2)  and  the  supremum  is  taken  for  all 

constants  a  and  0  satisfying  (37).  For  simplicity  we  put  r  = 
2  1/2 

(a^j  ♦  Pj)  .  For  the  coding  scheme  (36),  we  know  that 

Et|0j  -  8j|2]  •  <7„  (p2  .  .  <7 

Denote  by  Z2  =  ECZ2iy(Yj)]  and  Z2  =  ECZ2iycZj)].  Then  Z2  = 
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T  2^  =  ^12^11^1 


-1. 


12 


EClZg  -  22^3  =  a22 


2 


and 

El\Z^  -  22!^]  =  A<r“J. 

ft. 

By  (4)  we  have 

1(0, Y)  =  1(0, Yj)  +  h(Y2lYj)  -  h(Y2lY^,0). 

We  can  show  that 

Iie.Yj)  =  ^  log  (  )  ■  i  lo* 

h(Y2lYj)  =  ^  log  {27te  EC|a02  ♦  0  (0j  -  0j)  +  (Z^  -  22)!^]} 

and 

h(Y2lYj,0)  =  h(Z2lYj,0)  =  ^  log  {27te  E[|Z2  -  22!^]} 

=  2  ^0^  (27ieA<7jJ). 

2  2 

In  order  to  get  the  capacity  Cp(p^,P2>,  we  need  to  maximize 
9(a,«)  =  EC|a02  ♦  5  (0j  -  0j)  +  (Z2  -  22)1^3, 


which  can  be  written  as 

<p(o,0)  =  EC|o02  ♦  0  (0j  -  0j)|^3 

♦  20  EC(0j  -  QjXZg  -  22)3  ♦  EtIZg  -  221^3 
=  P2  -  20  EC0jZ23  ♦  Et|22  -  ^21^3 


(38) 

(39) 

(40) 

(41) 
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Since 


p|  -  ♦  ®22 


/,2  ^-2 
■  "^12^ 


P2  =  EC|0j  -  §^1^3  = 

B  can  run  on  the  interval 


^2^11  ^  ^  ^  ^  ^2^11 

Thus,  if  aj2  >  0  (resp.  <  0),  <pCa,B)  Is  maximized  when  B  = 

-1/2  -1/2 

-  ^2^11  ^  (reap.  B  =  ^2^11  ot  =  0,  and  the  maximum  value 

is 


P 


2 

2 


♦  a 


22 


2PjPj 


Combining  this  with  (38)  -  (41),  we  set 

2 

from  the  definition  of  Cp(p  ). 


\o 


-1/2 

11 


(22).  Eq. 


(23)  is  clear 

Q.E.D. 


Appendix 

PROOF  OF  THEOREM  2.  For  each  (P?,***,Pm)  such  that  Z  p^ 

1  N  n®i  n 

=  P^,  we  denote  by  Cp(Pj , • • • , pj)  (resp.  ^^(Pj , • • • , pj) )  the 
capacity  of  the  GC  with  (resp.  without)  feedback  under  a  constraint 

E[xJ]  aC  pj,  N  «  l.'--,N,  ■  (42) 

n  n 

It  is  known  that 
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4 


Co<P?.---.pS>  =  2  Jj  >“S  (  1  *  P^;"  )• 


In  order  to  prove  Theorem  2  it  suffices  to  show 


Co(Pi , • • * .Pn>  =  * 


For  any  pair  (0,X)  satisfying  (a.l)  and  (42)  we  have 

h(Z  |2,,-**,Z„  ,)  =  h(Z^)  =  J  log  (27reA^) 
nl  n~i  n^  n 


(43) 


and 


h(Y^|  Y,  .  •  •  •  ,Y„  ,)  ^  h(Y^)  sC  ^  log  [27ie  (pj  +  A^)]. 
n  1  n“i  n  4  n  n 


Therefore,  using  Lemma  1,  we  have 


1(0, Y) 


and  consequently 


\  J,  IP*  ( 1  *  p^;' ). 


2  2  '«*  ( 1  *  p^;" )  -  co<'’5----'"S> 

n=  1 


Thus  we  get  (43)  and  the  proof  is  completed. 


Q.E.D. 


Aoknovledgenent .  The  authors  would  like  to  thank  Professor 

C.R. Baker  for  a  number  of  helpful  discussions. 
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